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Abstract 

We consider discrete-time plants that interact with their controllers via fixed discrete alphabet sets 
in the absence of exogenous inputs. For this class of systems, we propose a general procedure for 
f*i constructing a sequence of finite state approximate models starting from finite length sequences of input 

D and output signal pairs. We show that the proposed construction satisfies desirable properties of p/u 

Ph approximations, thus leading to nominal deterministic finite state machine models that can be used in 

certified-by-design controller synthesis. We also show that the proposed construction satisfies a relevant 
^s^j 'completeness' property, and we conclude with directions for future work. 

1 Introduction 

o 

■ 1.1 Motivation 

Cyber-physical systems, involving tightly integrated physical and computational components, are omni- 
present in modern engineered systems 10 . These systems are fundamentally complex, and pose multiple 
challenges to the control engineer. In order to effectively address these challenges, there is an inevitable 
need to move to abstractions or model reduction schemes that can handle dynamics and computation in a 
unified framework. Ideally, an abstraction or model complexity reduction approach should provide a lower 
complexity model that is more easily amenable to analysis, synthesis and optimization, as well as a rigorously 
quantifiable assessment of the quality of approximation. This would allow one to certify the performance of a 
controller designed for the lower complexity model and implemented in the actual system faithfully captured 
by the original model, without the need for extensive simulation or testing. 
£SJ The problem of approximating systems involving dynamics and computation (cyber-physical systems) 

or discrete and analog effects (hybrid systems) by simpler systems has been receiving much attention over 
the past two decades [I 34 . In particular, the problem of constructing finite state approximations of hybrid 



X 



systems has been the object of intense study, due to the rampant use of finite state machines as models of 
computation or software, as well as their amenability to tractable analysis 29 and control synthesis [9 14 
(though tractable does not always mean computationally efficient!) 



ro 1.2 Overview of the Contribution 

In a previous effort [28], we proposed a notion of finite state approximation for 'systems over finite alphabets', 
basically plants that are constrained to interact with their feedback controllers by sending and receiving 
signals taking values in fixed, finite alphabet sets. We refer to this notion of approximation as a 'p/p 



approximation', to highlight the fact that is is compatible with the analysis 32 and synthesis [33] tools 
we had previously developed for systems whose properties and/or performance objectives are described in 
terms of p/p gain conditions. Note that the proposed notion of p/p approximation explicitly identified 
those properties that the approximate model needs to satisfy in order to enable certified-by-design controller 
synthesis. However, it did not restrict us to a particular constructive algorithm. 

In this paper, we propose and analyze a new constructive procedure for generating p/p approximations 
of a given plant and performance objective. In contrast to the state-space based construction presented as a 

*The author is with the Department of Electrical & Computer Engineering Department at the Johns Hopkins University, 
Baltimore, MD, 21218 (dtarraf@jhu.edu). Preliminary versions of this work were presented in [26p7] . 
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simple illustrative example in [28], which was specifically tailored to the dynamics in question, the present 
construction is widely applicable to arbitrary plants over finite alphabets provided that: (i) They are not 
subject to exogenous inputs, and (ii) their outputs are a function of the state only (i.e. analogous to strictly 
proper transfer functions in the LTI setting). 

Our proposed construction essentially associates states of the approximate model with finite length sub- 
sequences of input-output pairs of the plant. Since the underlying alphabets are finite, the set of possible 
input-output pairs of a given length is also finite. The resulting approximate models thus have finite state- 
space, and are shown to satisfy desirable properties of pj p approximations, rendering them useable for control 
synthesis. 



1.3 Related Work 

Automata and finite state models have been previously employed as abstractions or approximate models of 
more complex dynamics for the purpose of control design: We survey the directions most relevant to our 
work in what follows. 

One research direction makes use of non-deterministic finite state automata constructed so that their 
input /output behavior contains that of the original model (these approximations are sometimes referred to 



as 'qualitative models') [12 13 20 . Controller synthesis can then be formulated as a supervisory control 



problem, addressed using the Ramadge-Wonham framework [21 1122] . More recently, progress has been made 



in reframing these results [16, 17 in the context of Willems' behavioral theory and Z-complete systems 
35 . Our construction bears some resemblance to algorithms employed in constructing qualitative models. 
However, our notion of pj p approximation is fundamentally different from the notion of qualitative models, 
as it seeks to explicitly quantify the approximation error in the spirit of robust control. 



A second research direction, influenced by the theory of bisimulation in concurrent processes 15 18 
makes use of bisimulation and simulation abstractions of the original plant. These approaches, which typically 
address full state feedback problems, effectively ensure that the set of state trajectories of the original model 
is exactly matched by (bisimulation), contained in (simulation), matched to within some distance e by 
(approximate bisimulation), or contained to within some distance e in (approximate simulation), the set 
of state trajectories of the finite state abstraction [7j[l9|[23j[25] . The performance objectives are typically 
formulated as constraints on the state trajectories of the original hybrid system, and controller synthesis is a 
two step procedure: A finite state supervisory controller is first designed, and subsequently refined to yield 
a certified hybrid controller for the original plant 241. 

Other related research directions make use of symbolic models [2][8] , approximating automata [3j[5] , and 
finite quotients of the system [4||36[. 



Of course, the idea of using finite length sequences of inputs and outputs is widely employed in system 



identification 11 . However, the setup of interest to us is fundamentally different for three reasons: First, 
the dynamics of the plant are exactly known. Second, the data is complete (or can be completely generated) . 
Third, the data is exact and uncorrupted by noise. 

Finally, the presently proposed construction differs from our preliminary efforts reported in 1 30] , as it 
approximates the performance objectives as well as the dynamics of the systems, and moreover leads to a 
finite state nominal model with deterministic transitions. 



1.4 Organization and Notation 

We begin in Section [2] by reviewing the relevant notion of pj ' p approximation as well as basic concepts 
that will be useful in our development, We state the problem of interest in Section [3] We revisit a special 
structure in Section [4j We demonstrate its relevance to p/p approximations, and we address the related 
question of disturbance alphabet choice. We propose a constructive procedure in Section [5j and we show 
that the resulting approximate models satisfy the desired properties of p/p approximation in Section [6] We 
demonstrate further relevant properties in Section [7] highlighting the completeness of this construction. We 
conclude with directions for future work in Section[8l 

We employ fairly standard notation: Z+, K+ denote the non-negative integers and non-negative reals, 
respectively. Given a set A, A z + and 2 A denote the set of all infinite sequences over A (indexed by Z + ) and 
the power set of A, respectively. The cardinality of a (finite) set A is denoted by |„4|. Elements of A and 
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A z+ are denoted by a and (boldface) a, respectively. For a £ A z+ , a(i) denotes its i th term. For / : A — > B 7 
C CB, = {a £ A\f{a) £ C}. Given P C (U x ft) z + x (y x V) z + and a choice u Q € W z +, y Q £ y z + , 

P| Uoi y o denotes the subset of P defined as -P| Uo , yo = |(( u ' r )> (yi v )) £ P u = u a and y = y Q |- 



2 Preliminaries 

In our development, it is often convenient to view a discrete-time dynamical system as a set of feasible signals, 
even when a state-space description of the system is available. We thus begin this section by briefly reviewing 
this 'feasible signals' view of systems. We then present the recently proposed notion of p/p approximation 
specialized to the class of systems of interest (namely systems with no exogenous inputs), and we state two 
relevant control synthesis results. 



2.1 Systems and Performance Specifications 

Readers are referred to [32] for a more detailed treatment of the basic concepts reviewed in this section. A 
discrete-time signal is an infinite sequence over some prescribed set (or "alphabet"). 

Definition 1. A discrete-time system S is a set of pairs of signals, S C U z+ x y z + , where U and y are 
given alphabets. 

A discrete-time system is thus a process characterized by its feasible signals set. This description can be 
considered an extension of the graph theoretic approach [6] to the finite alphabet setting, and also shares 
some similarities with the behavioral approach [35] though we insist on differentiating between input and 
output signals upfront. In this setting, system properties of interest are captured by means of 'integral p/p' 
constraints on the feasible signals. 

Definition 2. Consider a system S C U I '+ x y z + and let p : U — > R and p : y — >• R be given functions. S 
is p/p gain stable if there exists a finite non-negative constant 7 such that 

T 

5^7P(u(*))-MlK*))>-°°- (!) 
- t=o 

is satisfied for all (u, y) in S. 

In particular, when p, p are non- negative (and not identically zero), a notion of 'gain' can be defined. 

Definition 3. Consider a system S C U z+ x y' l + . Assume that S is p/p gain stable for p :U — > R+ and 
p : y — > R+, and that neither function is identically zero. The p/p gain of S is the infimum 0/7 such that 
|7p is satisfied. 

Note that these notions of 'gain stability' and 'gain' can be considered extensions of the classical definitions 
to the finite alphabet setting. In particular, when U, y are Euclidean vector spaces and p, p are Euclidean 
norms, we recover I2 stability and I2 gain. We are specifically interested in discrete-time plants that interact 
with their controllers through fixed discrete alphabets in a setting where no exogenous input is present: 

Definition 4. A system over finite alphabets S is a discrete-time system S C U Ij+ x (y x V) z + whose 
alphabets IA and y are finite. 

Here u £ W z + represents the control input to the plant while y £ y z+ and v £ V z+ represent the sensor 
and performance outputs of the plant, respectively. The plant dynamics may be analog, discrete or hybrid. 
Alphabet V may be finite, countable or infinite. The approximate models of the plant will be drawn from a 
specific class of models, namely deterministic finite state machines: 

Definition 5. A deterministic finite state machine (DFM) is a discrete-time system S C W z + x y z + with 
finite alphabets IA, y , whose feasible input and output signals (u, y) are related by 

q(t + l) = /(«(*), u(t)) 
y(t) = g(q(t),u(t)) 

where t £ Z+, q(t) £ Q for some finite set Q and functions f : Q xU — >• Q and g : Q x U — >• y. 
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<2, / and g are understood to represent the set of states of the DFM, its state transition map, and its 
output map, respectively, in the traditional state-space sense. 

Finally, we introduce the following notation for convenience: Given a system P C U z + x (y x V) z+ and 
a choice of signals u Q £ U z + and y Q € y z+ , -P| Uo ,y denotes the subset of feasible signals of P whose first 
component is u Q and whose second component is y - That is 

^|u ,y = |fu, (y,v)) e P u = u and y = y |- 

Note that P| Uo ,y may be an empty set for specific choices of u Q and y Q . 



2.2 p/fi Approximations for Control Synthesis 

The following definition is adapted from [28] for the case where the plant is not subject to exogenous inputs, 
of interest in this paper: 



A, 



P 



Mi 



Figure 1: A finite state approximation of P 



Definition 6. (Adapted from Definition 6 in \2m) Consider a system over finite alphabets P C U z+ x (y x 
V) z+ and a desired closed loop performance objective 



T T 



t=0 T ^°T^0 

for given function /1 : V — > R. A sequence {M{\°°^ 1 of deterministic finite state machines Mi C (U x W) z + x 
(y x Vj x Z) 7j+ with Vj C V is a p/n approximation of P if there exists a corresponding sequence of 
systems {AJ^ l7 A* C Z z x W z + , and non-zero functions p& : Z z + — > M + , /ia : W z + — » K + , such that 
for every i: 

(a) There exists a surjective map ipi : P — > Pi satisfying 

^(P|u,y) QPi\u,y 

for all (u, y) e W z + x y z + , where Pi C U' z+ x(]/x Vi) z+ is ifce feedback interconnection of Mi and A, 
as shown in Figure^ 

(b) For every feasible signal (u, (y, v)) € P, we ftawe 

< < MM*))> ( 3 ) 

/or a/Z t G Z + , where 



(u, (yi,v ; )) = Vi((u, (y,v))J, 
(u, (yi +1 , v i+ i)) = ((u, (y, v))) . 
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(c) Aj is Pa/a*a gain stable, and moreover, the corresponding pa/ma gains satisfy 7,; > 7^+1- 

We complete our review of preliminaries by stating two results relevant to the problem of control synthesis. 
The first highlights the fact that a p/fi approximation of the plant together with an appropriately defined 
performance objective may be used to synthesize certified-by-design controllers for the original plant and 
performance objective: 

Theorem 1. (Adapted from Theorems 1 and 3 in \2$) Consider a plant P and a p/fi approximation {-Mi} 
as in Definition [fij If for some index i, there exists a controller K C y z+ x W z + such that the feedback 
interconnection of Mi and K, (M i; K) C W z + x (V; x Z) z + , satisfies 

T 

^ J2 T ^( w ( t )) - - T llPA (z{t)) > -00 (4) 

- t=0 

for some t > 0, then the feedback interconnection of P and K, (P,K) C V z+ , satisfies Q). 

The second provides a constructive value-iteration based approach for synthesizing a full state feedback 
controller for a given DFM in order to satisfy given performance objectives: 

Theorem 2. (Adapted from Theorem 4 in \33^ ) Consider a DFM M with state transition equation 

«(* + !) = f{q(t),u{t),w(t)), 

and let a : Qxt/xW^l be given. There exists a <p : Q —> U such that the closed loop system [M, cp) 
satisfies 

T 

^2 «>(*)) > (5) 

- t=o 

iff the sequence of functions Jk ■ Q — > K, k G Z +7 defined recursively by 

Jo = (6) 
J fc+ i = max{0,T(J fe )} 

where T(J(q)) = min max{— a(q, u, w) + J(f(q, u, w))}, converges, 
ueu wew 

Note that in particular, a gain condition such as Q, can be written in the form ^ as the outputs v and 
z of Mi are functions of the state of M and its inputs. 



3 Problem Setup 

Given a discrete-time plant P described by 

x(t+l) = f(x(t),u(t)) 

y(t) = g(x(t)) (7) 

v(t) = h(x(t)) 

where t € Z+, x(t) € R", u(t) € y(t) € 3^ and € V. No apriori constraints are placed on alphabet 
set V: It may be a Euclidean space, the set of reals, or a countable or finite set. U and y are given finite 
alphabets with \U\ = m and \y\ — p, respectively: They may represent quantized values of analog inputs and 
outputs, or may simply be symbolic inputs and outputs in general. Functions / : W L xW^ W 1 , g : K.™ — > y 
and h : M. n — > V are given. 

Also given a performance objective 

T 

sup fJt(v(t)) < 00. pi) 

Our goal is to provide a systematic procedure for constructing a p/fi approximation of P. This 
would allow us to subsequently simplify the process of synthesizing a controller K such that the closed loop 
system (P, K) satisfies the desired performance objective. 
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4 A Special Structure 



In 31 , we proposed a special 'observer-inspired' structure and used it in conjunction with a particular state- 



space based finite state machine construction in order to approximate and subsequently design stabilizing 
controllers for a special class of systems, namely switched second order homogenous systems with binary 
outputs. In this section, we consider a generalization of this structure, modified to allow for finite but non- 
binary sensor output alphabets. We first show in Section |4.1| that under mild conditions, this generalized 
structure ensures the existence of function if>i as required in property (a) of Definition [6] Second, we address 
in Section |4~2| the related question of minimal construction of the disturbance alphabet set W. 




Figure 2: A special structure that ensures existence of ipi 



4.1 Ensuring existence of ipi 

We begin with the following observation, noting that there is no loss of generality in assuming that finite set 
W with cardinality r + 1 is given by W = {0, . . . , r}: 

Proposition 1. Consider an alphabet set y with \y\ = p and a set W = {0, . . . , r}. For sufficiently large r, 
there always exists functions /? : y x y — » W and a : y x W — > y U {e} such that 

a(y,P(y,y)) = y 

for all y,y ey. 

Proof. By construction. Let r = p 2 — 1. Associate with every pair (j/i, y 2 ) £ y 2 a unique element of W, and 
define (3{y\,y2) to be equal to this element. Now define 

a(y',w) = ! V S "Sf'») = ™ . 
[ e otherwise 

We have a(y, /3(y, y)J = y for all y,y G 3^, as desired. □ 



G 



Consider the interconnection structure shown in Figure [2j and assume that the output y of system M, 
does not explicitly depend on the input y, to ensure that the interconnection is well-posed. We can now 
prove the following: 

Lemma 1. Consider a system P CU Z+ x(yxV) z+ wherelA and y are finite. Let W be a finite alphabet with 
sufficiently large cardinality, and let f3 : y x 3^ — > W and a^xW-JjU {e} be such that a(y, /3(y, y)) = y 
for all y,y G y. Now consider the system shown in Figure^ For any deterministic finite state machine 
Mi C (U x 3^) Z+ x (y x Vi) z + with fixed initial condition q Q , there exists a ipi : P — > Pj, where Pi is the 
interconnection of Mi and Ai, such that ipi is surjective and ipi ^P|u ,y ) C Pj| Uo .y - 

Proof. The proof is by construction. We begin by noting that for sufficiently large W, functions (3 and a with 
the desired property exist by Proposition 1 We also note that the condition a( y, j3(y, y)^j = y, Vy, § G y 
ensures that the output y G 3^ Z+ of P, matches the output y G of P for every choice of u G U z+ . Now 
consider : P — ► M, defined by ^nu, (y, v)J = ((u,y), (y, v)) G Mj, where (y, v) is the unique output 
response of Mi to input (u, y) for initial condition q Q . Also consider ip2.i ■ ipi, i(P) — > Pi defined by: 



^,i(((u,y),(y,v))) =(u, (y,v)) 



This map is well-defined and its image lies in Pj by virtue of the structure considered. Let ipi = ip2i ° i- 
Note that ipi is surjective since ip2,i is surjective and ip2~l{Pi) = i>i(P) by definition. Moreover, ipi(P\ Uo ,y ) C 



- p i|u , yo since 



^i(-P|u ,y ) - V'2,i(V'l,i(^|u ,y )) = ^2,i (((u OJ Yo), (y, v)) 6 Mi) C P| Uo ,y 

which concludes our proof. □ □ 

It follows from Lemma [l] that by restricting ourselves to approximations {M} with the structure shown 
in Figure [2j where Mi (for each i G Z + ) is an arbitrary finite state machine with fixed initial condition, 
property (a) of Definition [6] is guaranteed by construction, and we only need worry about constructing {Mi} 
to satisfy properties (b) and (c). 

4.2 Choice of Alphabet Set W 

Next, we turn our attention in this setting to the choice of alphabet set W: A set with minimal cardinality is 
desired, as the complexity of solving the full state feedback control synthesis problem, and hence the output 
feedback problem, grows with the cardinality of W, as seen in Theorem [2] Thus, in this section, we address 
the following question: What is the minimal cardinality of W for which we are able to construct functions 
/? and a with the property a(jj, f3(y, y)j = y? 

Lemma 2. Given a set y with \y\ = p. Let W* = {0, 1, . . .,p* — 1} be the smallest set for which there exists 
f3 : y x y ^>-W and aJxW-^jU {e} satisfying a(y, j3(y, y)j — y for all y, y G y. We have p* = p. 

Proof. The proof is by construction. Consider a map (3 : y x y — » W defined as shown in the Table [T] to be 
read as (3(y 1 ,y 1 ) = 0, /3(y 2 , yi) = 1, and so on. 

Now consider function aJxW->yu {e} defined as 



a(y',w) 



y iffi(y',y)=w 

e otherwise 



as in Proposition [T] By construction, each element of W appears exactly once in every row of the table. 

Thus a is a well-defined function, and it is straightforward to show by inspection that a(jj, (3(y,y)j — y for 

all y,y G y. Finally, note that when p* < p, some element of W would have to appear twice in each row of 
the table, thus rendering a ill-defined. □ 
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2/2 


2/3 


2/4 


■ V P 


2/1 







p-2 


p-3 . 


. 1 


2/2 


1 





p-1 


p-2 . 


. 2 


2/3 


2 


1 





p-1 . 


. 3 


2/4 


3 


2 


1 







2/p 


p-1 


p-2 


p-3 








Table 1: Definition of (3 : y x y -> W when | ^| = p 



5 Construction of Mj 

What remains is to construct a sequence of DFM {Mi} that, when used in conjunction with the structure 
described in Section[4]and shown in Figure[2j ensures that properties (b) and (c) of Definition [6] are satisfied. 
We begin by giving the intuition behind this construction in Section [O] before discussing the details of the 
construction in Section 15.21 



5.1 Inspiration for the Construction 

Consider a discrete-time SISO LTI system S described by 

x(t +1) = Ax(t) + Bu{t) 
y{t) = Cx(t) + Du(t) 



(8) 



where t £ Z+, x(t) £ K™, u(t) £ K, y(t) £ R, and A, B and C are given matrices of appropriate dimensions, 
and D is a given scalar. Assume that the pair (C, A) is observable and the pair (A, B) is reachable. Under 
these conditions, following a fairly classical derivation that is omitted here for brevity, we can express the 
state of the system at the current time in terms of its past n inputs and outputs as 



x(t) = [ A n O- 1 R-A n O~ 1 M ] 



where R = [B AB . . . A n 1 B] is the reachability matrix, 



y(t-i) 

y(t - n) 
u(t- 1) 

u{t — n) 



(9) 



O 



CA 11 - 1 

CA n-2 

c 



is a row permutation of the observability matrix, and M is the matrix of Markov parameters 

" D CB ... CA n ~ 2 B ~ 
'■• '■■ : 



M = 











CB 
D 



This observation suggests an approach for constructing a sequence of approximate models of S starting 
from finite length input-output sequence pairs of S: The states of the k th approximate model, Sk, are thus 
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those subsets of R 2k that constitute feasible snapshots of length k of the input-output behavior of S. In 
particular, for sufficiently long snapshots (i.e. for k = n), the approximate model converges to the original 
one. Indeed, when the current state of S n is 

x(t) = [y(t -l),...,y(t- n),u(t -l),...,u(t- n)]' 

its output in response to input u(t) is uniquely defined and given by 

y(t) = Cx(t) + Du(t) 

where C = [ CA n O~ 1 CR — CA n O~ 1 M ] . Likewise its next state in response to input u(t) is uniquely 
defined and given by 

x(t + 1) = [Cx(t) + Du(t),y(t - 1), . . . , y{t - n + 1), u(t),u(t - 1), . . . , u{t - n + 1)]' 
= Ax(t)+Bu(t) 

for appropriately defined^ matrices A and B. 
We wish to highlight three points: 

1. If systems S and S n are identically initialized, meaning that their initial states obey 

x(0) = [ A n O- 1 R - A n O- 1 M ] 4(0) 

their outputs will be identical for any choice of input u £ . An alternative approach for comparing 
the responses of S and S n without explicitly matching their initial states is by considering an "approx- 
imation error" A with the structure shown in Figure [2] (P then corresponds to S and Mi corresponds 
to S n ). In this setup, S n is additionally given access to the outputs of S, allowing it to estimate the 
initial state: State x(t) of S n can thus be thought of as its the best instantaneous estimate of the state 
x(t) of S. At time steps t < n — 1, the state set of S is refined as follows 

x(0) £ R 2n , 

4(1) £ {v £ R 2n \v(l) = y{Q),v n+1 = u(0)}, 

x{2) £ {v £ K 2 >(1) = y(l),v(2) = y{0),v n+1 = u(l),v n+2 = u(0)} 

and so on. At time steps t > n, x(t) is uniquely defined by the expression in ^. The C 2 gain of A, 
defined here as the infimum of 7 > such that the inequality 

T 

mf o 5> 2 HO||| 2 -|Ki)|| 2 >-oo 
- t=o 

holds, compares how well the outputs match after the transient (i.e. after S n is done estimating the 
initial state of S): Note that the outputs of S and S n will exactly match for all times t > n, the £2 gain 
of A in this case is zero, and S n can be thought of as an "exact" model of S. The internal structure of 
A thus has a nice intuitive interpretation that may not have been as transparent to the readers when 



we introduced it in 31 



2. Output y(t) cannot explicitly depend on input y(t), otherwise S n can trivially match the output of S 
at every time step, rendering the comparison meaningless. 

3. There does not exist a one-to-one correspondence between the states of S and S n : Since the kernel of 
matrix [ A n O~ 1 R — A n O~ 1 M ] in (j9j) has non-zero dimension, one state of S can correspond to 
several states of S n , rendering this description inherently redundant. 



While the use of S n as an alternative to the LTI model in QSJ) of S is not justifiable here, this exercise 
suggests a procedure for constructing approximations of systems over finite alphabets: In that setting, hi 
and y are finite leading to approximate models with finite state-spaces. 



lr The exact expression for A and B is not relevant to the discussion, and is thus omitted for brevity. 
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5.2 Details of the Construction 

Given a plant over finite alphabets as in ^ and a performance objective as in ([2]), we construct the corre- 
sponding sequence {Mj} as follows: For each i 6 Z+, M$ is a DFM described by 

q(t + l) = fMt),u(t),y(t)) 

y(t) = 9Mt)) (10) 
v(t) = hi(q(t)) 

where t E Z+, £?(*) € Q i5 u(*) € W, y{t) e y, y(t) e y, and v(t) e V l . 
State Set: The state set is 

Q l = Qi,FUQi,iU{q $ ,q } 

where 

Qi t F - Set of final states. This is where the state of Mj evolves for t > i. 

Qi j - Set of initial states. This is where the state of Mj evolves for 1 < t < i. 

gj - Impossible state. This is where the state of Mj transitions to when it encounters an input-output pair 
that does not correspond to plant P. 

q Q - Initial state. This is the fixed initial state of Mj at t = 0. 

More precisely, using the shorthand notation f u {x) to denote f(x,u), we have 

> Qi,F C y i X U\ q = (yi, . . . , y t , m, . . . , m) € Qi, F if 3x e E" such that 

y% = 9\xo} 



Vi-l 

I) 



2 = j(/u,_ 1 o/»,W) (11) 



2/1 



g(fu 2 o ■ • • o f Ui (x f) 



> Qi,J = Qi,J,i U . . . U Qj,j,j where Qj,/^ Cj J xtf and # = . . . , y jt m, . . . , Uj) € Qij,j if 3x D E R n such 
that 



2/j = 9\x 0j 
Vj-x = g[fu 3 (x )j 



yi 



g(fui °~-°fu s (a?o)) 



(12) 



Transition Function: The transition function /j : Qj x W x y — > Qi is defined as follows: 
t> For q = (yi, . . . ,yi,ui, . . . ,Ui) E Q*,f, we define 

*t v \-{ 9 = (2/> 2/i, ■ • -,yi-i,u,ui, . . .,Ui-i) if $ € Qi,F 
\ gj otherwise 

> For q = q , we define 



f(q ,u,y) 



q® otherwise 
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q = (y, ■ • .,yj,u,ui, ...,Uj) if q G Qtjj+i U Q iii? 

<70 otherwise 



t> For q = (yi,...,yj,ui,...,Uj) € Qijj, we define 

f{q,u,y) -- 

> For 90, we define /(<70, u, y) = q$ for all it € U and y ej. 

Output Functions: We begin by associating with every q £ Qi & subset X(q) of R™ defined as follows: 
t> For 9 = (j/i,...,^,^,...,^) € Q^jr, let 



X = {x E R n \x satisfies ([IT])} (13) 

and define 

X(q) = f Ul o...of u .(X ) (14) 
> For q = (y%, ... ,yj,ui, ... ,Uj) e Qi,i,j, let 



X = { x € R"|x satisfies Q} (15) 

and define 

X(g) = / Ul o...o/„.(J5f ) (16) 

> Define 

jT(g)= IT' q=q ° ( 17 ) 

We can also associate with every q G Qi a subset F(q') of 3? defined as 

y( ? ) = 5 (A-(«)) (is) 

We are now ready to define the output function gi : Qi — > y as 

q (q) = { V f ° r S ° me Vey ' if ^ = (19) 

1 y for some y € ^(?), otherwise 

The output function hi : Qi ^ Vj is defined as 

J h(argmax x< z X ( q )fJ'(h(x))), q € Qi,F U Qi,i U {q a } 
hi(q) = < > \ (20) 

I ft( argmin xe i&n (j,(h(x)) ) , q = q$ 

Output Set: The output set Vi is defined as 

V s : = (J hi{q) 

Remark 1. We conclude this section with a few observations: 

1. The output of Mi corresponding to a state q is chosen arbitrarily among the feasible options. The 
possibility of error is accounted for in the gain 7$ of Aj. 

2. Our definition of the performance output function hi assumes that the map \i : K. — > M has a well-defined 
minimum and maximum. This places some mild restrictions on the original problem. 

6 p/fi Approximation Properties of the Construction 



In this Section, we show that the construction of {Mi} proposed in Section 5.2 together with the structure 
analyzed in Section|4] indeed allows us to meet the remaining two properties of Definition[6j namely properties 
(b) and (c). 
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p 



y 



u 



Mi 



M, 



i+l 



Vi 



Vi+1 



Vi+i 



Figure 3: Interconnection of P, Mi and Mj+i 



6.1 Meeting the Condition on the Performance Objectives 



Proposition 2. Consider a plant P as m jf?j), a performance objective as in Q), and a DFM Mi constructed 
following the procedure given in Section \ 5.S\ for some i > 1. Consider the interconnection of P and Mi 
Figure [3| Let x{t) and qi{t) be the states of P and Mi, respectively, at time t. We have 



as shown in 



x(t) G X(q(t)) for all t > 0, for any choice of u G W z +, a;(0) G 

Proof. Pick a choice u G U z + and x(0) G E n . At t = 0, g(0) = g G and 
x(0) G X(g(0)). For 1 < t < i, we can write 



by construction. Thus 



X 



(«(*)) = { 



= < x G 



l |x = f u (t-i) o . . . o f u m)(x ) for some x D G K." that satisfies 



(12 )}. 



Thus x(t) G X(q(t)) since it can indeed be written in that form for some x , namely the initial state of P, 
x a = x(0), and x D satisfies (12). For t>i, we can write 

X(q(t)) = |a; G R n \x = / u (t-i) ° ••• fu(t-i){ x o) f° r some a; D G M" that satisfies (11) 

Again we have ir(i) G A(q(<)), since can be written as x(i) = f u /t-i) ° • • • fu(t-i)( x (t ~ *))) an< ^ x(t — i) 
satisfies (111. Finally, we note that our argument is independent of the specific choice of u G W z+ , and is 
also independent of the initial state of P, which concludes our proof. □ 



Proposition 3. Consider a plant P as i n a performance objective as in Q), and a DFM Mi constructed 
following the procedure given in Section \5.S\ for some i > 1. Consider the interconnection of P and Mi as 
shown in Figure^ We have fi(v(t)) < [i(vi(t)) for all t>0, for any choice of u G 14 Z+ , x(0) G R n . 

Proof. Pick a choice u G U z+ and x(0) G M". It follows from Proposition [2] that the corresponding state 
trajectories of P and Mj satisfy x(t) G for all t G Z+. We have q(t) 7^ (70 for all t, since Mj is driven 

by a, feasible pair (u,y) of P in this setup. Let x~i(t) = argmax x( zx(q i (t))l JI '{h{x{t))). It follows from (201 that 

/*(«(«)) = n(h(x(t)) < »(h(xi(t)f) - 

One again, noting that our argument is independent of the specific choice of u G W z + , and of the initial state 
of P, we conclude our proof. □ 
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Proposition 4. Consider a plant P as in a performance objective as in M), two DFM Mi and Mi + \ 
constructed following the procedure given in Section \5.S\ for some i > 1 . Consider the interconnection of P, 
Mi and Mi+i as shown in Figure^ We have fi(vi+i(tj) < fi(vi(t)) for all t > 0, for any choice of u £ U z + , 

i(0)el". 

Proof. Pick a choice u £ U L+ and x(Q) £ R n . Let q^t) and qi+i(t) denote the states of Mj and M i+1 , 
respectively, at time t. For 1 < t < i, we can write 



X(qi(t)) = |.t e R n \x = fu(t-i) ° • • • /ufojf^o) for some x € R." that satisfies (12) j 



and 



X 



(q i+ i(t)) = £ R n \x — /„(t_i) o . . . o f u (o)(x ) for some x G e R" that satisfies (12l|. 



Since X(qi(t)) = X(q i+ i(t)), it follows from (20) that ^,(vi + i(tj) = fi(vi(t)) for all 1 < t < i. For t > i, we 
can write 



X 



(qi(t)) = ja: G R"|a; = f u (t-i) ° ••• fu{t-i){ x o) f° r some x e R n that satisfies (11) j 



and 



(<Zi+i(£)) = |x € R"|a: = f u (t-i) ° ■■■ fu{t~i-i)( x o) for some £ € R™ that satisfies (11)|, 



with « replaced by i + 1 in (111. Thus X(qi + i(t)) C Letting iri(t) = argmax x€ x(q i (t))f'{h{x(t))) 

and a;i+i(t) = argmax x ^x(q i+1 (t))t L (h(x(t))), it follows from (201 that 

Kt>i+i(t)) = < KKMt))) = M(«i(*))- 

Finally, we note that our argument is independent of the specific choice of u £ U z + , and is also independent 
of the initial state of P, which concludes our proof. □ □ 

We can now state and prove the main result in this Section: 



Lemma 3. Consider a plant P as in Q), a perft 



ormance 



objective as in Q), and a sequence of DFM {M{\ 
constructed following the procedure given in Section 5.2 For every (u, (y,v)) £ P, we have 



for all t £ Z + , where 



/*(«(*)) </*(«i+i(*)) <M(0i(*)), © 

(u,(y i ,v i ))=^((u, (y,v))), 

(u, (yi+i, Vi+i)) = ip i+1 ((u, (y, v))) . 

Proof. Recall from the proof of Lemma[T]that map ipi was constructed as tp2,i ° ipi,i where ipij : P — > Mi is 
defined by 

^i,J(u ,(yo,v)J = ((u ,y ), (yi,Vj)) e Mi. 

Here (y, v^) is the unique output response of Mi to input (u ,y ) for initial condition qi(0). Also recall that 
i>2,i ■ i>i,i(P) — > Pi was defined by: 

^,,(((u„jo), (y 4 ,v*))) = (u , (y ,v )) 

Thus it suffices to show that for any (u Q , (y Q , v)) £ P, the outputs of Mi and Mj+i, (y^, v^) and (y»+i, 
respectively, in response to input (u ,y ), satisfy the desired condition. This follows directly from Proposi- 
tions |3] and H □ 
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6.2 Meeting the Condition on the Gains 

Next, we show that under some mild assumptions, the proposed construction of {Mi} meets property (c) of 
Definition [6j We begin with some definitions: 

Definition 7. A function fi : W — > R+ is positive definite if fJ,(w) > for all w €E W and n{w) = iff 
w = 0. 



Definition 8. Consider a sequence of DFM, {Mi}, constructed as described in Section 5.2 The sequence 
{Mi} is output-nested if for every i £ Z+, and for every 

q = (yi, ■ ■ ■ ,Vi,Ux, . . .,Ui)' G Qi, 

5 = (yij ■ ■ ■ ,yi,yi+i,ui, ■ ■ -,Ui,ui+i)' g Qt+i 

wii/i |y (g) | > 1, t/ie equality <?i+i(q) = <7i(<?) holds. 

We can now prove the following: 
Proposition 5. Consider a plant P as in Q), a performance objective as in HQ, and two DFM Mj and 



Mi + \ constructed following the procedure given in Section 5.2, for some i > 1. ^IssMTne i/ioi i/ie sequence 
{Mi} is output nested, and consider the interconnection of P , Mi and M i+ i as shown in Figure [3| Let 
Wi(t) — (3(y(t),yi(t)) and u>i+i(t) = (3(y(t),yi + i(t)) for f3 defined as in Table^given in the proof of Lemma 
Q For any choice of u £ 14 Z+ , initial state x(0) G W l of P, and fiA positive definite, we have 

Ma(iu;+i(*)) < MA 

Proof. Pick a choice u € U z+ , ir e E™, and : W — >• R+ positive definite. Let ^(t) and g i+ i(t) denote 
the states of Mj and Mj+i, respectively, at time i. By arguments similar to those made in the proof of 
Proposition |4j omitted here for brevity, we have 

X(q i+1 (t))=X{ qi (t)), forl<t<i 
I(? i+1 (t))Cl( gj (i)), tort>i 

It thus follows that 

Y(q i+1 (t))=Y( qi (t)), ioil<t<i 



Y(q i+1 (t))CY( qi (t)), fort>i 

For 1 < t < i, since {Mi} is output nested, we have jji{t) = Vi+i(t), thus Wi{t) = w i+ i(t) and fj,^(wi(t)) = 
HA(wi+i(t))- On the other hand, for t > i we can consider two cases: First, if |y(gj+i(t))| = 1, we have 

\Y(q i+l (t))\ = l =► y{t)=y i+1 (t) 
Wi + i(i) = 

jUA(^t+l(*)) = 
=*> MA(Wi+l(t)) < flA(Wi{t)) 

where the first implication follows from Proposition [2] and the third and fourth follow from the positive 
definiteness of /ja- Second, if |F(gi + i(i))| > 1, we have 

\Y{q i+1 (t))\ > 1 => ft+i(« i+ i(t)) = <7i(ft(t)) 

y»+i(*) = Vi(t) 

=S> ttfi+l(*) = Wj(f) 

jUA(^i+l(*)) = VA(Wi(t)) 

where the first implication follows from the assumption that {Mi} is output-nested. We thus conclude that 

(iA(Wi+l(t)) < MA (»«»(*))) V* > 0. 

What is left is to note that our argument was independent of the choice of u € U z + , x(0) € M. n , and positive 
definite function //a- d 
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We are now ready to state and prove the first main result in this section: 

Lemma 4. Consider a plant P as in Q), a performance objective as in and a sequence of DFM {Mi} 
constructed following the procedure given in Section \5.2\ Assume that is positive definite and the sequence 
{Mi} is output-nested. For any i > 1, the gains of Aj and A,-+x satisfy 7.; > 7^+1. 

Proof. Recall from the proof of Lemma[T]that map ipi was constructed as ip2,i where ipij : P — > Mj is 
defined by 

^l.iKUo, (yo,v)) = ((u ,y ), (yi,Vi)) e M h 

where (y, Vj) is the unique output response of Mi to input (u ,y ) for fixed initial condition qi(0). Also 
recall that if>2, i '■ i>i,i(P) P was defined by: 



^,,(((u„j ), (yj,Vi))J = (uo, (y ,v )) 



Thus it suffices to show that for any (u Q , (y Q , v)) € P, the outputs of Mi and Mj+i in response to input 
(u ,y ), (yi,Vj) and (yi+i, Vi+i), respectively, satisfy the desired condition. 

Let ji be the gain of Aj. Pick a choice of (u Q , (y Q , v)) € P, and let (y,, Vj) and (yi+i, v»+i) be the unique 
outputs of Mi and M i+1 , respectively, in response to input (u ,y ). It follows from Proposition [5] that 

fj,A{wi+i(t)) < ma («>*(*)), Vt -ma (ifi+i (*))> -Ma («;<(*)), Vt 

7»MA(«o(*)) - MA(Wi+l(i)) > JtPA(u (t)) - fJ,A{Wi(t)), \ft 
T T 

=> ^2liPA(Uo(t)) ~ Ma(«J»+i(*)) > ^7iPA(Wo(*)) - Ma(^»(*))) Vr 
i=0 t=0 
T T 



y^7»PA(Mo(*)) - MA(«>t+i(*)) > inf ^7i/°A(w (t)) - HA(wi(t)), VT 

t=0 ~~ 4=0 

T T 

inf ^7*Pa(u (*)) - MA(wj+i(i)) > inf 2J JiPA(u (t)) - Ma(w»(*)) 
- t-o 



Letting 7j_|_i = inf 7 such that 



T 

Inf^^K^)) -Ma(w <+ i(*)) > 
- t=0 



we have 7»+i < 7j. Since this argument holds for any choice of (u Q , (y Q , v)) 6 P, we have 

7i+i = inf{7 i+ i} < 7,, 

where the 'inf is understood to be taken over all possible choices of feasible signals of P. □ 

Note that Lemma [4j while effectively establishing a hierarchy of approximations, did not address the 
question: When is 7$ finite? A straightforward way to guarantee that is to require p\{z) > for all z. While 
this may be meaningful in a setup where we have no preference for particular choices of control inputs (since 
u = z in our proposed structure), this may be too restrictive in general. We thus propose, in what follows, 
a readily verifiable sufficient condition to ensure that 7^ < 00 without requiring p& to be strictly positive. 

Proposition 6. Consider a plant P as in Q), a performance objective as in M), a positive definite function 
j-iA ■ VV — > R+, and a DFM Mi constructed following the procedure given in Section 5.2, for some i > 1. Let 
7i be the gain of the corresponding error system Aj, and define a new 'output' function for M i7 e : Qi — » K + , 
defined by 

' if\Y{q)\ = \orq = q % 



e(q) 



max/iAfw) otherwise 
wew 
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Let 7i be the infimum of 7 such that Mi satisfies 

T 

1 1 1 



jnf V^aHO) - e(*) > -00- 



t=o 

FFe ftowe 7^ < 7,-. 

Proof. Pick a choice of (u Q , (y Q , v)) 6 P, consider the interconnection of P and Mi as shown in Figure [3] 

let and q(t) be the states of P and Mj, respectively , at time t, and let e(t) be the corresponding 'new' 

output of Mi. When \Y(q(t))\ = 1, we have x(t) £ X(q(t)) by Proposition [2j and thus y(t) = j/^i), to(i) = 

(in the corresponding A,-), and e(t) = u(w(t)). When |Y(g(f))| > 1, we have e(i) = maxpA(w) > MA(w(i)). 

toew 

It thus follows that e(t) > /j, A (w(t)) for all t > 0, and we can write 

A*a(«>(*)) < e(t), Vt =>> -A*a(«j(*)) > -e(t), Vt 

JiPA(u {t)) - («>(*)) ^ lip&{u (t)) - e(t), Vt 

T T 

^7iPA("o(i)) - HA(w(t)) > 2^7iPA(w (t)) - e(i), VT 

t=0 t=0 
T T 



y^7»PA(^o(*)) - Ma(w(*)) > inf y^7jp A (^ ft)) - e(t), VT 

t=0 _ t=0 

T T 

inf X^aKW) _ fj, A (w(t)) > inf ^mKW) - e W 



T>0 Z -^ t>0 

_ t=0 ~ t=0 

Letting 7, = inf 7 such that 

T 

|nf ^7PaK(*)) -^aO(*)) > -00, 
- t=o 

we have 7^ < 7,. Since this argument holds for any choice of (u Q , (y OJ v)) € P, we have 

7, = inf{7i} < j u 

where the 'inf is understood to be taken over all possible choices of feasible signals of P. □ 

Proposition [6] essentially establishes an upper bound for the gain 7$ of A, , verified by checking that Mj 
alone satisfies a suitably defined gain condition. This is readily verifiable using standard techniques from 
linear programming, or alternatively from network flow problems: Readers are referred to [32] for the details. 
However, here, we are not interested in computing 7,. Rather, we are simply interested in verifying that it 
is finite. As such, we can simplify the problem even further, by requiring a suitably defined gain condition 
to be met by a new DFM, referred to as the 'O-reduction' of M». 

Definition 9. Consider a DFM M t C (WxJ) z + x(y x V ! xM + ) z + and a choice p A : U -> E+. LetU = {u € 
U\p&{u) = 0}. The O-reduction of Mi, denoted by M i} is a new DFM, M l C (U x y) z + x (y x V l x R+) z + , 
obtained from Mi by restricting the first input of Mi to hi. 



Remark 2. It follows from Definition ^^that a state q of Mi, q = (yi, . . . , yj, u%, . . . , uj) € Qi (for some 
j G {0, . . . , i}), is a state of Mi iff Uk € U for all k € {1, . . . , j}. The number of states of Mi can thus be 
significantly lower than that of Mi . 

We can now propose the following sufficient condition to ensure that 7* is bounded: 

Lemma 5. Consider a plant P as in Q), a performance objective as in Q), a positive definite function 
//A : VV —¥ K. + , and a DFM Mi constructed following the procedure given in Section 5.2, for some i > 1. Let 



1G 



7,: be the gain of 
in PropositionuA 



the corresponding error system A, . Define a new 'output ' function e : Qi 
and let Mi be the O-reduction of Mi . If Mi satisfies 



for Mi 



T 

|^ S7PA («(*)) -e(t) > - 
- t=o 



00 



(21) 



for some finite 7, then 7$ is finite. 



Proof. Construct a graph (or a network flow problem) corresponding to Mi by associating with every state 
transition of Mi a cost, namely '7/Ca( u ) — e' defined by the input u that drives the transition and the output 
e associated with the beginning state of the transition. M, satisfies (21 ) iff every cycle in the corresponding 
network flow problem has non-negative total cost (the proof of this statement is omitted for brevity - readers 
are referred to 32 for the details). In particular, 7$, the infimum of 7 such that (21 1 is satisfied, is infinite 



iff there exists a cycle in M,-, driven entirely by inputs in U, and such that e ^ for at least one state along 
the cycle. Thus, it suffices to verify that Mj satisfies ( plj ) for some finite 7 to ensure that 7* < 00, from 
which we can deduce that 7, is finite by Proposition [6j □ 



7 Further Properties of the Construction 

In this Section, we prove one additional property of the given construct: Intuitively, we show that if a 
deterministic finite state machine exists that can accurately predict the sensor output of a plant after some 
initial transient, then our construct recovers it. While the resulting DFM generated by our construct is not 



expected to be minimal (due to the inherent redundancy in this description, see the discussion in Section 5.1 1, 
this property suggests that our construct is well-suited for addressing analytical questions about convergence 
of the approximate models to the original plant. 

Theorem 3. Consider a plant P as in |?j), a performance objective as in pj|), a positive definite choice of 



jj. a : W — > M.+, and a sequence {Mi} constructed following the procedure given in Section 5.2, with {71} 
denoting the gains of the corresponding approximation errors {Aj}. Assume there exists a deterministic 
finite state machine M with fixed initial condition, such that the A resulting from the interconnection of P 
and M as shown in Figure^ has gain 7 = 0. Then 7,;* = for some index i* . Moreover, if {Mi} is output 
nested, 7^ = for all i > i* . 

Proof. Assume a DFM M with the stated properties exists, and let w(t) be the output of the system A 
constructed by interconnecting P and M as shown in Figure [2| By assumption, we have 

T 

I < 00 



inf 0.p&(u(t)) - ma («>(£)) > -00 <^ sup HA{w(t)) ■ 
- {=0 1 - t=0 



Since W is finite, the cardinality of /xa(W) is also finite, as is that of the state set of M. Thus there must 
exists a time T* such that /j,^{w(i)) = for all t > T* , or equivalently w(t) = for all t >T* (by the positive 
definiteness of /za)- Now let i* — T* , and consider the corresponding DFM Mi* in the constructed sequence. 
We claim that \Y(q)\ = 1 for every q G Qi»,F- The proof is by contradiction: Indeed, suppose that \Y(q)\ > 1 
for some q = (y%, . . . ,yi*,Ux, . . . ,Ui*)' € Qi*.F- Thus, there exists an input sequence, namely it(0) = u±, 
u(l) = U2,..., u(T* — 1) = Ui* with two corresponding feasible sensor outputs of P given by y(0) = yi, 
y(l) = y 2 ,. . ., y(T* - 1) - W . , y(T*) = y 1 and y(0) = Vl , y(l) = y 2 ,. . y{T* - 1) = w . , y{T*) = y" where 
y' 7^ y". Since M has fixed initial condition, its response to the input sequence is fixed, and it thus follows 
that w(T*) 7^ for some run, contradicting the fact that w(t) — for all t >T* . This cannot be, and hence 
|F(g)| = 1 for all q € Qi*,F- It follows from this and Proposition [2] that yi*(t) = y(t) for every t > T* , and 
thus 7i» = 0. When {Mi} is output nested, we can immediately conclude from the above and Lemma|4]that 
7i = for alH > i*. □ 
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8 Conclusions and Future Work 



In this paper, we revisited the recently proposed notion of p/p approximation. We proposed a general 
procedure for constructing a sequence of finite state machines for systems over finite alphabets that are not 
subject to exogenous inputs. We showed that the resulting constructs satisfy the three required properties of 
pj p approximations. We also showed that they exhibit a 'completeness' property, in the sense that if a finite 
state machine exists that can perfectly predict the sensor output after some transient, then our construct 
recovers it. Finally, we showed that the cardinality of the minimal disturbance alphabet that can be used in 
this setting equals that of the sensor output alphabet. 
Future work will focus on two directions: 

1. At the theoretical level, it is clear from the construct that the problem of approximation and that of 
state estimation under coarse sensing are closely intertwined. We will thus focus on understanding the 
limitations of approximating certain classes of systems by these approximations, or at a more basic 
level, the limitations of reconstructing the state under coarse sensing. 

2. At the algorithmic level, the work will focus on improving the efficiency both at the implementation 
level as well as the algorithmic development level. In particular, we will look into refining this procedure 
by developing a recursive version. 
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